Optimize UInt256 logical shifts with a portable funnel shift - #109

Summary

UInt256.Lsh/Rsh are rewritten as a single portable funnel shift. There is no architecture gate and no second implementation.

• The carry term is written (lo >> 1) >> (63 - bitShift). That equals lo >> (64 - bitShift) but yields 0 when bitShift is 0, so whole-word counts need no separate path. The (n % 64) == 0 switch, the Lsh64/Lsh128/Lsh192 and Rsh64/Rsh128/Rsh192 helpers and NativeLsh/NativeRsh are all removed.
• Only the word offset is still branched on, as a compare chain rather than a switch. A 4-way switch here compiles to a jump table reached by an indirect jmp; that predicts perfectly when a benchmark fixes the shift count per case but not on a mixed count stream, and Dynamic PGO cannot reorder it.
• The result is written in one store of the width callers read it back at.
• Negative-count, out-of-range and aliasing semantics are unchanged.

Head history: 8baac47 (the original X86Base.X64-gated implementation) is preserved, main is merged in, and 71be7f6 replaces the implementation. The net diff against main is the funnel rewrite only.

Why the gated implementation was replaced

The previous approach kept origin/main's body as a second implementation behind X86Base.X64.IsSupported, but its own fast path contained no intrinsics, so the gate bought nothing except a duplicate body to maintain.

More importantly, its wordShift switch arms build new UInt256(...), which at the time wrote the result as four 8-byte stores. A 32-byte load cannot be store-forwarded from four 8-byte stores; it waits on L1, roughly ten cycles. Most of this type loads a UInt256 as a single Vector256 (AddAvx2, LessThanAvx2, ToBigEndian), as does the EVM stack. Measured against 392d749, that path was 1.53-1.58x slower than base for a caller reading the result as 32 bytes, while looking 0.81-0.91x (faster) for a caller reading .u0/.u3. A benchmark that consumes the result narrowly cannot see it.

That is consistent with this PR's own earlier finding of +1.3% on the eth_call replay alongside a large microbenchmark win.

main has since changed the four-argument constructor to write plain limbs (#103/#104), so this specific gap has narrowed: against 3a03caf the old implementation measures 0.86-0.96 across the same four shapes, with no regression. The store-width effect itself did not go away, it became main's behaviour, and 25 sites in UInt256.cs still read a UInt256 as a Vector256. SetLimbs therefore stores through Unsafe.As rather than returning a UInt256 (struct promotion splits the latter back into limb stores), and falls back to limb stores where Vector256 is not hardware accelerated, because there the callers read limbs too and the software Vector256.Create is out-of-line calls.

Measurements

Headline: this saves between roughly half a cycle and roughly twelve cycles per shift call, depending on how the caller reads the result. The large, reliable win is that callers which load the result as one 32-byte value stop paying a store-to-load forwarding stall. Everything else is a modest tightening worth one to four cycles.

Zen 5 (9950X), .NET 10, Tier-1 + Dynamic PGO, baseline main at 3a03caf, on a Solidity-idiomatic shift mix (selector/address/byte extraction, power-of-two scaling, plus out-of-range counts).

Cycle figures assume 4.3 GHz. Nothing regresses in any shape.

What the shapes mean. "Reads 1x32B" is a caller that loads the shift result as a single Vector256 - AddAvx2, SubtractAvx2, LessThanAvx2, ToBigEndian, and the EVM stack. "Reads 4x8B" is a caller that touches individual limbs. "Throughput" is independent shifts back to back; "dependency chain" feeds each result into the next shift, so it measures latency. Real code sits between the two, and the 32-byte row is the common one for this type.

Where the 11-12 cycles come from. A 32-byte load cannot be store-forwarded from four 8-byte stores; it waits on L1. main writes the result as limb stores, so a Vector256-reading caller stalls. Writing one 32-byte store removes the stall, which is why that row is the outlier and why the two consumer widths are reported separately.

Method and caveats

Both variants live in one process; each round measures baseline and candidate with rotating order; per-round ratios are reduced to a median, and that is repeated across 9 independent processes with the median taken again. An A/A control (baseline in the candidate slot) sits at 0.999-1.000 in every cell, min 0.994, max 1.005.

• Absolute times include an out-of-line call in both arms, identical on each side. Deltas are therefore the meaningful figure and the ratios understate the change to the shift body itself.
• The 32-byte cells are sensitive to code layout, which shifts store-to-load forwarding. Medians reproduced closely across two separate 9-process aggregations (0.440/0.392 and 0.440/0.394 for throughput), but individual processes have shown as much as 0.937 on one and 1.251 on another. Do not draw conclusions about these two rows from a single process, in either direction.
• This is one machine and one synthetic shift distribution. It does not tell you how often each caller shape occurs in a real block.

Code size

JitAsm, Tier-1 + PGO, x64:

Neither body emits a jump table or an indirect jump. Under FullOpts main emits two out-of-line Lsh192 calls and this emits none. For reference, the superseded 8baac47 body was 510/512 B at Tier-1.

Correctness

• Full suite: 576,401 passed / 0 failed with intrinsics on, and 576,400 passed / 1 skipped with DOTNET_EnableHWIntrinsic=0 (the skip is the pre-existing hardware-accelerated hash test, unrelated to shifts).
• Focused shift suite: 467 tests, 0 skipped, also verified under DOTNET_EnableAVX2=0 and DOTNET_EnableAVX512F=0.
• CI matrix is green on Windows, Linux, macOS and ubuntu-24.04-arm, in HWIntrinsics and NoHWIntrinsics, debug and release, plus the zkEVM variant. ARM64 exercises the limb-store branch of SetLimbs, where Vector256.IsHardwareAccelerated is false.
• Cross-checked out of tree against a BigInteger oracle: every count in [-300, 300] plus int.MinValue, int.MaxValue and +/-2^20, over 72 values, for Lsh, Rsh and both aliased forms.

The test file no longer gates negative non-word counts behind X86Base.X64.IsSupported; with one implementation there is nothing left to skip, and it now covers every count in [-260, -1] on every target. Negative counts are pinned rather than designed: the pre-1.6.1 path produced those results by violating its own Debug.Assert(n < 64), so the behaviour is release-build fallout, not a contract. Pinning it means it can be retired deliberately instead of by accident, and defining n as unsigned would be a reasonable follow-up.

Outstanding

• The seeded 497-record eth_call replay needs re-running against 71be7f6. The result recorded below (+1.3% median, "held for rework") measured 8baac47, which is no longer the implementation, and the baseline it used has also moved.
• Also measured and rejected: an AVX2 branchless word shift (vpermd with a sign-masked index, 133 B) and an AVX512VL vpermt2q variant. The AVX2 form wins the throughput shapes 0.60-0.75x but loses the dependency-chain shapes, and averaged over 24 points the portable scalar funnel matched the AVX512 version, so no ISA gate was added. Three ways of sharing the funnel across the word-offset arms (goto case fall-through, and two switch-selection forms) are 30-40% smaller and all slower, because the duplication is specialisation: wordShift == 3 needs one shl and sharing forces all four funnels on every call.